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(57) ABSTRACT 

Multiple digital watermarks, each of which has different 
characteristics, are embedded in a document. The charac- 
teristics of the various watermarks are chosen so that each of 
the watermarks will be affected in a different manner if the 
document is subsequently copied and reproduced. 

The detection process or mechanism reads each of the 
watermark and compares their characteristics. While wear 
and handling may change the characteristics of the digital 
watermarks in a document, the relationship between the 
characteristic of multiple digital watermarks in a document 
will never-the-less give an indication as to whether a docu- 
ment is an original or a copy of an original. 
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MULTIPLE WATERMARKING TECHNIQUES 
FOR DOCUMENTS AND OTHER DATA 

RELATED APPLICATIONS 

The present application is a continuation in part of appli- 
cation Ser. No. 09/234,780 filed Jan. 20, 1999 now aban- 
doned is now and which is a continuation of application Ser, 
No. 60/071,983 filed Jan. 20, 1998. Both applications Ser. 
Nos. 60/071,983 and 09/234,780 are assigned to the 
assignee of the present application. 

FIELD OF THE INVENTION 

The present invention relates to steganography and more 
particularly to the use of watermarks to determine the 
authenticity and history of a particular document or image. 

BACKGROUND OF THE INVENTION 

Steganographic and digital watermarking technologies 
are well know. For example see U.S. Pat. No. 5,636,292 and 
the extensive references cited therein. Also see copending 
patent applications Ser. No. 08/327,426 which was filed Oct. 
21, 1994 and copending application Ser. No. 08/436,134 
which was filed May 8, 1995. 

The technology for inserting digital watermarks in images 
and the technology for reading or detecting digital water- 
marks in images is well developed, well known and 
described in detail in public literature. Furthermore, there 
are commerciaUy available products which include pro- 
grams or mechanisms for inserting digital watermarks into 
images. For example the commercially available and widely 
used products "Adobe Photoshop" which is marketed by 
Adobe Corporation of San Jose California and "Corel Draw" 
program which is marked by Corel Corporation of Ontario 
Canada, include a facility for inserting digital watermarks 
into images. 

The technology for making high quality copies of docu- 
ments is widely available. The technical quality of scanners 
and color printers has been increasing rapidly. Today for a 
relatively low cost one can purchase a high quality scanner 
and a high quality color printer. Thus, it is becoming 
increasingly easy to duplicate documents. The ability to 
create high quality copies has created a need for technology 
which can differentiate between original documents and 
copies of the original. 

It is known that watermarks can be used to help differ- 
entiate genuine documents from copies. However, the prior 
art techniques for using digital watermarks to differentiate 
genuine documents from copies have serious limitations. 
The present invention is directed to an improved technique 
for using steganography and digital watermark technology 
to facilitate differentiating original documents from copies 
of the original. 

The present invention can also be used for various other 
purposes such as to embed multiple types of information in 
a single document or to provide watermarks which enable 
documents to perform special functions. 

SUMMARY OF THE INVENTION 

With the present invention multiple digital watermarks, 
each of which has different characteristics are embedded in 
a document. The characteristics of the two watermarks are 
chosen so that each of the watermarks will be affected in a 
different manner by what may subsequently happen to the 
document. 

The detection process or mechanism reads the two digital 
watermarks and compares their characteristics. While wear 



and handling may change the characteristics of the indi- 
vidual watermarks, the relationship between the character- 
istic of the two watermarks will never-the-less give an 
indication as to whether a document is an original or a copy 

5 of an original. 

For example according to the present invention two 
digital watermarks in a document may have different energy 
levels. The absolute energy level of a digital watermark in an 
original image may be decreased if a document is subject to 

10 wear. Likewise the energy level of the digital watermark in 
an image may be decreased if an image is scanned and 
reprinted on a color printer. However, the relationship 
between the energy level of the two digital watermarks will 
be different in an image that has been subject to wear and in 

35 a reproduced image. Likewise if two digital watermarks are 
introduced into an image where the bit pattern used to 
construct the digital watermarks have different patterns, the 
ratio between the signal to noise ratio of the watermarks will 
be different in an original subject to wear and in a copy 

20 generated by scanning the original and printing the scanned 
image. Other characteristics of multiple digital watermarks 
can also be used to differentiate original documents from 
copies. 

25 BRIEF DESCRIPTION OF THE FIGURES 

FIG. 1 shows the paths that a document and a copy may 
follow. 

FIGS. 2 A and 2B show a fine grain and a course grain 
30 watermark. 

FIGS. 3 A and 3B show a geometrically linear and a 
geometrically random assignment of pixels to a bit in a 
digital watermark. 

FIG. 4 illustrates a fourth embodiment of the invention. 

35 

DETAILED DESCRIPTION OF PREFERRED 
EMBODIMENTS 

The problem of differentiating an original document from 

4Q a copy is made more difficult in situations where the original 
document is subject to being handled, worn, folded and 
otherwise damaged. Many original documents such as iden- 
tification documents and currency are extensively handled. 
The wear to which such documents are subjected reduces the 

45 quality of images on the document and therefore reduces the 
quality of any information embedded in the document using 
conventional steganographic techniques. 

With the present invention a number of different water- 
marks are embedded in a document. Each of the watermarks 

50 embedded in the document, has different characteristics. All 
watermarks are somewhat affected when a document is 
subjected to wear, and all watermarks are somewhat affected 
when a document is duplicated by being scanned and 
reprinted. However, the magnitude of the effect caused by 

55 being scanned and reprinted on watermarks with certain 
characteristics is much greater than the effect on watermarks 
with different characteristics. Likewise, wear and handling 
of a document affects watermarks with certain characteris- 
tics much more than it affects watermarks with different 

60 characteristics. 

Thus, if multiple watermarks with different characteristics 
are inserted into a document, it is possible to differentiate a 
copy from an original document that has been subjected to 
wear by examining the ratios of characteristics of the 

65 watermarks in the image being examined. 

In order to print a document on a color printer, the 
document is put through a transformation from a color space 
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such as the RGB color space to a different color space such 
as the CMYK (cyan, magenta, yellow, black) color space. 
Such transformations are well know. For example see chap- 
ter 3 entitled "Color Spaces" in a book entitled "Video 
Demystified, A handbook for the Digital Engineer", Second 
Edition, by Keith Jack, published by Harris Semiconductor 
and Hightext Publications of San Diego, Calif. 

When an image is transformed from one color space to 
another color space, noise is introduced into the image. 
Among the reasons for this is the fact that each color space 
has its own distinctive gamut (or range) of colors. Where the 
gamut of two color spaces overlap, the conversion from one 
color space to another color space can in theory be precise. 
However, there will be some areas which are in the gamut 
of one color space not in the gamut of another color space. 
Such situations definitely introduce noise into the conver- 
sion process. Even in areas that are in the gamut of two color 
spaces, conversion from one color space to another color 
space introduces noise because of such things as round off 
errors. The present invention takes advantage of the fact that 
if an original is copied and then a copy is printed, the image 
on the printed copy will have gone through several conver- 
sions to which the original will not have been subjected. 

For example, the conversions to which a copy may be 
subjected are: 

1) a document to RGB conversion (i.e. scanning the docu- 
ment into the computer), 

2) a RGB to CMYK conversion, 

3) a CMYK to copy conversion (i.e. printing the document). 
Any characteristics of the two digital watermarks that will 

be affected differently by the additional conversion process 
to which copies are subjected can be used to differentiate 
copies from an original. Since the two watermarks with 
different characteristics are affected in a different manner by 
the additional conversion step, a comparison of the charac- 
teristics of the two watermarks in a document being exam- 
ined will indicate if the document is an original (which has 
not gone through the additional conversions) or a copy 
which has gone through the additional conversions. While 
the characteristics of each watermark will have been 
changed by wear and by the copying process, the compari- 
son between the characteristics of the two watermarks will 
still be able to differential a copy from an original. 

Four embodiments of the invention are described below. 
Each of the embodiments utilizes two watermarks in a 
document. The differences between the two watermarks in 
the document are as follows: 

In the first embodiment: 

First watermark: Has fine grain 

Second watermark: Has a course grain 

In the second embodiment: 

First watermark: Has geometrically linear assignment of 
pixels 

Second watermark: Has geometrically random assign- 
ment of pixels. 
In the third embodiment: 
First watermark: Has low power 
Second watermark: Has higher power 
In the fourth embodiment: 

Fist watermark: uses standard RGB to HSI and HSI to 

RGB transformations 
Second watermark is biased before being transformed 

from HSI to RGB. 
FIG. 1 shows the steps to which documents and copies are 
typically subjected. In the normal course, a document 10 



may be subjected to handling and wear 11 resulting in a 
worn document 10A. Document 10 may also be scanned as 
illustrated by box 12. The scanning produces a digital image 
which can be printed as illustrated by box 13. The printed 

5 image may be subjected to handling and wear 14 resulting 
in a copy 10B. It is noted that the document 10 may also be 
subject to handling and wear prior to the scanning operation 
12. The task to which this invention is directed is the task of 
differentiating the worn document 10A from the copy 10B. 

10 The document 10 includes an image (not explicitly 
shown) which has two digital watermarks inserted therein. 
In the first embodiment of the invention, the first watermark 
has a fine grain and the second watermark has a course grain. 
The grain of the two watermarks is illustrated in FIG. 2. FIG. 

15 2 A shows the grain of the first watermark and FIG. 2B shows 
the grain of the second watermark. The first watermark uses 
blocks of 9 pixels (a 3 by 3 block). Each of the pixels in each 
9 pixel block has its gray value changed by the same amount. 
For example FIG. 2A shows that the first 9 pixel block has 

20 its gray value increase and the second 9 pixel block has its 
gray value decreased. The amount of increase and the 
selection of blocks that is increased and decreased is con- 
ventional. 

As shown in FIG. 2B, the grain of the second watermark 
25 is in blocks that are 6 pixels by 6 pixels or 36 pixels. All of 
the pixels in each 36 pixel block are changed by the same 
amount. 

In the original document 10, the two watermarks have a 
power ratios of 1 to 1. After wear and handling, the power 

30 of the first watermark will be degraded somewhat more than 
the power of the second watermark. For example, as illus- 
trated in FIG. 1, after document 10 is subjected to handling 
and wear, a detector which reads the watermarks might find 
that the power ratio of the water marks is 1 to 2. 

35 If the document 10 is scanned and the resulting digital 
image is printed to make a copy of the document 10, the ratio 
of the power of the watermarks will be affected much more 
than the effect of handling and wear. For example as 
illustrated in FIG. 1, the power ratio of the watermarks may 

40 be 1 to 10, thereby allowing one to differentiate the worn 
original document 10A from the copy 10B. 

It is noted that the mechanism for inserting watermarks 
into an image is well known as is the technique for reading 
a watermark and using correlation techniques to determine 

45 the signal to noise ratio (i.e. the power) of a watermark. 
FIGS. 3 A and 3B shown an alternative technique for 
implementing the present invention. In the second embodi- 
ment of the invention, the two watermarks inserted into the 
image on a document have different patterns of assigning 

50 pixels to the bits of the number which the watermark 
represents. The first watermark utilizes a geometrically 
linear assignment of pixels to each bit. For example FIG. 3 A 
shows an image that has 500 by 500 pixels. Considering a 
watermark with 50 bits, each bit of the watermark would 

55 have 5000 pixels assigned to represent that bit. A linear 
assignment could have each fifth bit in each row (100 bits 
per row) and each fifth row (50 rows) assigned to each bit 
of the watermark. Thus 5000 pixels would be assigned to 
each bit in a very orderly or linear manner. 

60 In the second watermark the pixels would be assigned to 
each bit in a random manner as shown in FIG. 3B, Each bit 
in the watermark would still have 5000 assigned bits; 
however, the pixels would be a random location over the 
image. Naturally it should be understood that FIGS. 3 A and 

65 3B illustrate how pixels are assigned to one bit of the 
watermark. The other bits of the watermarks would have 
pixels assigned in a similar manner. 
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Similar to the first embodiment of the invention, the First calculate: 

watermark with a linear assignment of pixels and the water- if |< O r=0.5 then M=I (I+S) 

mark with a random assignment of pixels would be affected . - , n c t u \aj c to 

differently by handling and wear on the original document 1 > * en = + 

than they would be by being scanned and reprinted. 5 m=2I-M 

The third embodiment of the invention described herein if S=0 then R=G=B=I and H=180° 

utilizes watermarks which have different power levels. Han- Then calculate R, G and B as follows: 

dling and wear as contrasted to scanning and printing would a ) if H<60 then R»M 

affect a watermark with a low power level differently than a if H<12Q meQ Rsm+((M _ m) / ((120 -H)/60)) 

water mark with a high power level. Watermarks with 1Q 

different power levels can be inserted into a document in " H<240 then R=m 

order to practice the present invention utilizing commer- if H<300 then R=m+((M-m)/((H-240/60)) 

cially available programs such as Adobe Photoshop or Corel otherwise R=M 

Draw. In the Adobe Photoshop and Corel Draw programs, b ) if H<60 then G=m4<(M-m)/(H/60)) 

the power or intensity of the watermark can be adjusted by . fp 1fin , r \a 

setting a simple control setting in the program. 15 11 H<i * u men U=M 

The fourth embodiment of the invention introduces dif- if H <240 then G-m+((M-m)/((240-H_J60)) 

ferent characteristics into two watermarks by modifications otherwise G=m 

made to one of the watermarks during the initial step during c) if H<120 then B=m 

which the watermarks are introduced into an image. The if H<180 men B =m+((M-m)/((H- 120/60)) 

operation of the fourth embodiment can be explained as 20 . - u n , 

shown in FIG. 4. First as illustrated by equation 1 there is a 11 H<3U0 tnen B ~ M 

conversion from RGB to HSI as is conventional. This is otherwise B«m+((M-m)/((360-H)/60)) 

illustrated by equation 1. As illustrated by equation 2, the Next the values which represent a second watermark are 

first watermark is inserted into the image in a conventional used t0 calculate a second set of RGB values designated 

manner by modifying the I value in the HSI representation 2 5 RGB2 - m order to calculate RGB2, the values of H and S are 

of the image using the first watermark values (designated as modified so that they are slightly biased toward a particular 

WM1 A). A first RGB value designated RGB(l) is then color designated HI and SI New values for H and S are 

calculated using a conventional transformation designated T. calculated as follows: 

As indicated by equation 3, the second watermark WM2 is ( Note > H1 must between 0 and 360, SI must be non- 
then biased toward a particular color and the biased water- 30 negative and can be between 0 and 1 and X is a value 
mark is then combined with the HSI values and transformed between 0 and 1) 

to a second set of RGB values designated RGB(2). Finally Calculate new values for H and S as follows: 

as indicated by equation 4, the values RGB(l) and RGB(2) If H>H1 then H=H-(H-Hl)x 

are combined to form the watermarked image designated else H=H+(H1-H)x 

RGB(F). 35 If S>S1 then S=S-(S-Sl)x 

The transform used to go from RGB to HSI color space else S=S+(Sl-S)x 

(indicated in equation 1 in FIG. 4) can be anyone of a variety Next add the second watermark to the values of HSI and 

of known other techniques. For example, the RGB to HSI transform these values to the RGB color space as indicated 

conversion can be one of the techniques explained in the by equation 3 The transformation from HSI color space to 

above referenced text book such as the following: (which 40 RGB color space is done as previously indicated, 

assumes that RGB and Intensity have a value range of 0 to Finally as indicated by equation 4 in FIG. 4, the final RGB 

I and that Red equals 0°): value (designated RGBF) is calculated by combining the 

First calculate: values of RGB1 and RGB2. This combination can be done 

M=max (R,G3) in a variety of known ways. 

m=min (R,G,B) 45 It is noted that in the above example the difference 

r=(M-R)/(M-m) between the transformation used for the first and the second 

gm(M-G)/M-n) watermarks involves biasing the values of H and S. 

b=(M-B)/(M-m) Alternatively a wide variety of different changes could 

Then calculate I, S, and H as follows: also be made - ^ ke ? 10 this fourth embodiment of the 

a) I=(M+M)/2 50 mvent i° n * s that in effect a different transformation is used 

b) if M«m then S=0 and H=180 for the first and the watermarks. 

^ t^- r\ < c r\A m \//\A^ m \ While four embodiments of the invention have been 

11 I<or=U.5 then 5=(M-m)/(M+m) , , , , 

f t n < *u c_/ivj \u4 \a \ shown herein it should be understood that many other 

f R M th H fioX- ? characteristics and attributes of a digital watermark could be 

f i-Ti u u *n /o u\ 55 used t0 P ractice tne Present invention in addition to the 

if G«M then H-60 (2+r-b) characteristics and attributes described herein. Furthermore 

if B=M then H=60 (4+g-r) other known digital watermarking techniques can be used 

if H>or«360 then H=H-360 together with and applied to the digital watermarks used for 

if H<0 then H=H+360 the present invention. It is also noted that while in the above 

The first watermark in inserted into the RGB values in a 60 examples only two watermarks were used, in some situa- 

conventional manner by modifying the I value of appropri- lions one could use three, four five or more watermarks. That 

ate pixels so as to combine the watermark A values with HSI is, the embodiments of the invention specifically described 

values. This is indicated by equation 2 in FIG. 4. Next as herein utilize two watermarks. It should be understood that 

indicated by equation 3 in FIG. 4, the HSI values are any number of watermarks could be utilized in like manner, 

converted to RGB values using a transform T. The trans- 65 Furthermore while the embodiments shown herein utilize 

form "T" can be conventional and it can for example be done two separate watermarks, the two watermarks used to prac- 

as follows: tice the present invention could be combined into one 
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watermark which has a plurality of separate identifiable and 
measurable characteristics. 

It is noted that while the present invention utilizes mul- 
tiple watermarks with different characteristics to differenti- 
ate original documents from copies of the original, one can 5 
also utilizes multiple watermarks with different characteris- 
tics for other reasons. Documents may include multiple 
similar watermarks in addition to the watermarks which 
have different characteristics according to the present inven- 
tion. As used herein, in general, the term "document" refers 10 
to a physical entity. 

While the present invention has been described with 
respect to four specific embodiments of the invention, it 
should be understood that various changes in form and detail 
could be made without departing from the spirit and scope is 
of the invention. The scope of the present invention is 
limited only by the appended claims. 

We claim: 

1. A method of creating a watermarked image which 
comprises the steps of: 20 

modifying an HSI (hue, saturation, Intensity) representa- 
tion of an image to imbed a first watermark in said 
image and create a first watermarked image, 

transforming said first watermarked image to the RGB 
(red, green, blue) color space, biasing the values which 25 
represent a second watermark toward a particular color, 

modifying said HSI values to imbed said biased second 
watermark in said image and create a second water- 
marked image, 3Q 

transforming said second watermarked image to the RGB 
color space, and 



combining the values of said first and second water- 
marked images to create a final watermarked image. 

2. The method recited in claim 1 wherein said biasing is 
toward a particular color. 

3. A method of differentiating copies of an original 
document from the original document, said document con- 
taining a first digital watermark which has a first set of 
characteristics and a second digital watermark which has a 
second set of characteristics, said method comprising the 
steps of: 

reading said first and second watermarks from said origi- 
nal document and comparing the resultant values to 
generate a first set of results, 

reading said first and second watermarks from said copy 
of said original document and comparing the resulting 
values to generate a second set of results, and 

using differences between said first and second sets of 
results to differentiate an original document from a 
copy of said original document. 

4. The method recited in claim 3 wherein said first digital 
watermark has a first energy level and said second water- 
mark has a second energy level. 

5. The method recited in claim 3 wherein said first 
watermark has a first bit pattern and said second watermark 
has a different bit pattern. 

6. The method recited in claim 3 where both said original 
document and said copy have been subjected to wear. 
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